feat: update to newest ftw-tests-schema #209
Conversation
|
@theseion Looks like we have conflicts on this one... |
|
Depends on the new scheme changes. |
|
Requires coreruleset/ftw-tests-schema#7 |
- add new `retry_once` option - add test case for `retry_once` - make test infrastructure more modular
- add flag to stop test run on first test failure - fix statistics to record at test, not stage level
The log file was the last remaining setting that could not be configured from the command line.
The default read timeout is too short for some of the more expensive tests.
- always print test ID on failure; especially useful with --show-failures-only - use the same indentation for all output messages
theseion
changed the title
|
I think this is ready. It currently uses a fixed commit from This PR has grown quite a bit because many refactorings came in handy and I opted to adjust the schema on-the-fly where I thought it made sense. There's a draft PR for CRS where I've made all the necessary changes to support the code here: coreruleset/coreruleset#3369. With that PR, all tests for both nginx and httpd succeed (including all previously disabled and commented tests). Note that Coraza support will come a bit later with coreruleset/coreruleset#3377. It would be great if I could also get feedback from @jcchavezs and @christiantreutler-avi on how these changes will affect you. @jcchavezs Note that I opted to drop the |
|
Ugh, massive. Ok, will take some time to parse this. |
|
Well, honestly, is too big to understand cross implications between things. I'll approve and 🤞 |
retry_onceoptionretry_once